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DETAILED ACTION 



Response to Amendment 

1. The amendments, filed 04/14/2009, have been entered and made of record. Claims 1-4, 6-1 1, 
13-15 and 17-40 are pending. 



Response to Arguments 

2. Applicant's arguments with respect to 1-4, 6-11, 13-15 and 17-40have been considered but 
are moot in view of the new ground(s) of rejection. 



Claim Rejections - 35 USC § 103 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

4. Claims 1,3-4,6-11, 13-15, 17-32, 34-35, 37 and 39-40 are under 35 U.S.C. § 103(a) as being 
unpatentable over Gvili, "Depth Keying" SPIE Vol. 5006 (2003) in view of Luo, US 
2002/0093670, and further in view of Clarke, US 4,591,842. 

Regarding claim 1 , Gvili discloses a method for differentiating between foreground 
objects and background objects within a scene being captured through an image capture 
device (a novel depth video camera), comprising: 

emitting a ray of light from a light source toward an object of the scene (generating a 
light wall, Para 3.1, pp 566-567); 
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opening an aperture cover allowing access to a sensor of the image capture device for 
reflected light from the light source (deploying a fast image shutter in front of the 
CCD chip, page 567, lines 3-4); 
- closing the aperture cover after a set time (the shutter is precisely controlling the 
exposure time of the CCD, page 567, lines 17-18), the predefined amount of time 
corresponding to a maximum distance traveled by the light (a real-time trimap is 
generated for each frame based on the original depth matte, page 569, lines 22-23); 
and 

generating a depth mask identifying objects within a foreground region and a 
background region of the scene (foreground objects can be generated by setting the 
depth measurement window, page 568, section 3.2) based upon the light captured 
during the set time (a real-time trimap is generated for each frame based on original 
depth matte, page 569, 4th paragraph), the depth mask identifying objects within the 
foreground region with a first bit value (full value pixels representing foreground 
pixels in the color channel) and identifying objects within the background region with 
a second bit value (zero value pixel representing background) (page 564, 6 th 
paragraph). 

Gvili does not specifically teach the adjusting an image capture device parameter 
according to bit values of the depth mask for one or more of a sequence of captured image 
frames of the scene, wherein the image capture device parameter is selected from one of 
brightness, exposure or gain. 
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However, Luo does teach the adjusting an image capture device parameter according to 
bit values of the depth mask for one or more of a sequence of captured image frames of the 
scene, wherein the image capture device parameter is selected from brightness (the subject 
emphasizing step may include the step of altering the background by reduced brightness to 
produce a spotlight effect on the main subject) ([0016], Luo). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have included the subject emphasizing method as taught by Luo into 
Gvili's digital camera, as the subject content analysis and image modifications are performed 
automatically, and artifacts due to errors in content analysis are not noticeable and 
objectionable ([0018], Luo). 

Gvili and Luo do not specifically teach applying the adjusted image capture device 
parameter so that the captured image frames is adjustable independently for both of the 
objects within the foreground region and the objects within the background region. 

However Clarke does teach applying the adjusted image capture device parameter so that 
the captured image frames is adjustable independently for both of the objects within the 
foreground region and the objects within the background region (controlling the colors of a 
raster graphic system in which the background or foreground behaviors can be independently 
changed) (3:1-4 and 4:55-5:4, Clarke). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have included color adjusting step as taught by Clarke into Gvili and 
Luo's digital camera, as the combination could providing an improved method for 
controlling the colors of a raster graphic system in which the memory requirements are 
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decreased without decreasing the number of behaviors the system can display (2:64-68, 
Clarke). 

Regarding claim 3, Gvili discloses the light source is configured to emit infrared light (IR 
laser diodes, page 567, lines 15-16). 

Regarding claim 4, Gvili discloses a method operation of opening an aperture cover 
allowing access to a sensor of the image capture device (the shutter is precisely controlling 
the exposure time of the CCD, page 567, lines 17-18) includes, receiving reflected light from 
the objects within the foreground region (light reflected from every object inside the depth 
measurement window, page 568, second paragraph of section 3.2). 

Regarding claim 6, Gvili discloses a method operation of adjusting image capture device 
parameters according to bit values of the depth mask prior to capturing a subsequent 
corresponding image of the scene (ability to change the parameters of the depth window 
according to the scenario's need, page 568, section 3.2, first paragraph) includes, determining 
an optimal amount of light based upon the depth mask; and adjusting the aperture cover to 
allow the optimal amount of light into the image capture device (normalized depth of pixel 
can be calculated, page 567, lines 7-11). 

Regarding claim 7, Gvili discloses the image capture device parameters are adjusted 
through mechanical adjustments (depth key setting can be automatically set, page 568, third 
paragraph of section 3.2). 

Regarding claim 8, this claim recites same limitations as claim 3. Thus it is analyzed and 
rejected as previously discussed with respect to claim 3 above. 
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Regarding claim 9, this claim recites same limitations with features quite substantially 
similar as claim 1 . Thus it is analyzed and rejected as previously discussed with respect to 
claim 1 above. 

Regarding claim 10, Gvili discloses the method operation of generating a depth mask of 
the scene from data defining the image of the scene includes segmenting the foreground and 
background regions of the scene (automatically segmentation of the scene is possible, page 
568, third paragraph of section 3.2). 

Regarding claim 11, Gvili discloses the data defining the image of the scene includes 
pixel data where each pixel is tagged with distance information (calculate color distances 
between neighboring pixels, page 570, section d). 

Regarding claim 13, Gvili discloses the image capture device is selected from the group 
consisting of a digital camera, a web cam, and a camcorder (a novel depth video camera, 
page 564, line 4). 

Regarding claim 14, Gvili discloses the displaying a portion of the image of the scene 
having adjusted pixel values (a new matte is generated by combing the information from both 
the depth and the color frames, page 569, fourth paragraph and also page 571, first paragraph 
of section 4). 

Regarding claim 15, Gvili discloses the portion of the image of the scene is an image of a 
participant for use in an interactive gaming application (man-machine interactions, page 572, 
and second paragraph of section 5). 
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Regarding claim 17, this claim differs from claim 9 only in that the claim 9 is a method 
claim whereas claim 17 is an apparatus. Thus the apparatus claim 17 is analyzed and rejected 
as previously discussed with respected to claim 9 above. 

Regarding claim 18, Gvili discloses the depth mask is a bit mask having a first logical 
value (full value pixels) assigned to represent the foreground objects and a second logical 
value (zero value pixels) assigned to represent the background objects (full value pixels 
representing foreground pixels in the color channel and zero value pixel representing 
background) (page 564, 6 th paragraph). 

Regarding claims 19, this claim recites same limitations as claim 4. Thus it is analyzed 
and rejected as previously discussed with respect to claim 4 above. 

Regarding claim 20, Gvili discloses each logic element is one or a combination of 
hardware (i.e. shooting lens, depth sensor, as well as a camera) and software (i.e. a depth 
map, depth measurement window, and scene segmentation) (pp. 568-569, section 3.2 depth 
key setting) 

Regarding claim 21, Gvili discloses the image capture device is a video capture device (a 
novel depth video camera, page 564, line 4). 

Regarding claim 22, Gvili discloses the depth logic is further configured to periodically 
provide a depth mask for a sequence of video frames captured by the video capture device 
(step f of the process for iteration of alpha values, page 570). 

Regarding claim 23, Gvili discloses the image capture device setting is adjusted through 
one of a mechanical or electrical adjustment (depth key setting can be automatically set, page 
568, third paragraph of section 3.2). 
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Regarding claim 24, Gvili discloses the image capture logic is further configured to 
adjust each pixel of image data of the scene (step d of the process shown the color distance 
calculation is performed for each pixel of the image data of the scene)( step d of the process 
for iteration of alpha values, page 570). 

Regarding claim 25, this claim differs from claim 1 only in that the claim 9 is a method 
claim whereas claim 25 is an apparatus. Claim 25 further differs from claim 1 in that the 
limitations "a computing device" and "a display device" are additionally recited. Gvili 
teaches a computing device (a Pentium 3 machine, page 572, section 5) and a display (a 
video camera, page 564, line 4). Thus the apparatus claim 25 is analyzed and rejected as 
previously discussed with respected to claim 1 above 

Regarding claim 26, Gvili discloses the computing device is a game console (game input 
device, page 572, section 5). 

Regarding claim 27 and 28, these claims recite same limitations as claims 22 and 23, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 22 and 23 above. 

Regarding claim 29, Gvili discloses the video capture device is a webcam (ZCam is used 
to generate depth keying during live broadcasts, page 568, line 5 and page 572, section 5). 

Regarding claims 30 and 32, these claims recite same limitations as claim 11. Thus they 
are analyzed and rejected as previously discussed with respect to claim 1 1 above. 

Regarding claim 3 1 , Gvili discloses the scene image data includes an image of a person, 
the image of the person being incorporated into a video game for interaction therein (camera 
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using depth-sensing technology are used in man-machine interactions) ( second paragraph of 
Summary, page 572). 

Regarding claim 34, note the discussion of claim 1 above. Gvili does not disclose 
adjusting the captured image frames facilitates tracking of the objects within the foreground 
region by reducing brightness of the background region. However, Luo does teach adjusting 
the captured image frames facilitates tracking of the objects within the foreground region by 
reducing brightness of the background region (the subject emphasizing step may include the 
step of altering the background by reduced brightness to produce a spotlight effect on the 
main subject) ([0016], Luo). 

Regarding claims 35, 37 and 39, these claims recite same limitations as claim 34. Thus 
they are analyzed and rejected as previously discussed with respect to claim 34 above. 

Regarding claim 40, Gvili discloses the foreground and background regions define two 
layers of the scene (a foreground layer and a background layer); wherein the scene includes 
additional layers (multi-layered keys); and wherein the method operation of adjusting pixel 
values is performed for data defining the image corresponding to objects within any one of 
the layers of the scene (second paragraph on page 565 and 5 th paragraph on page 566, Gvili). 
5. Claim 2 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Gvili in view of Luo 
and Clarke, and further in view of Tuomi, US 7,061,507. 

Regarding claim 2, note the discussion of claim 1 above. Gvili, Luo and Clarke do not 
teach storing the depth mask in memory of the image capture device. However, Tuomi 
teaches storing the depth mask in memory of the image capture device (provided a Z-buffer 
1902 for storing the Z-values relating to the depth of the pixel; see 12:15-36, Tuomi). 
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It would have been obvious to one of ordinary skill in the art at the time this invention 
was made to have used a memory as taught by Tuomi as modified by Gvili, Luo and Clarke 
so that it provide multiple buffers for storing information (12:15-17, Tuomi). 
6. Claims 33, 36 and 38 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Gvili 
in view of Luo and Clarke, and further in view of Podoleanu, US 6,769,769. 

Regarding claim 33, note the discussion of claim 1 above. Gvili, Luo and Clarke do not 
teach adjusting a focus parameter of the image capture device according to the bit values of 
the depth mask, so that the objects within the foreground region are in focus. However, 
Podoleanu does teach adjusting a focus parameter of the image capture device (an optical 
mapping apparatus) according to the bit values of the depth mask, so that the objects within 
the foreground region are in focus (Podoleanu teaches a method utilizes an optical mapping 
apparatus where the parameters of the said synchronizing procedure of the optical coherence 
depth adjusting means and focusing adjusting means, parameters which are the range, initial 
position and velocity of the focus adjusting means for a given depth range and velocity of the 
optical coherence depth scanning means, wherein each optical coherence image in either loop 
is validated by the brightness and regularity of the confocal image or one of the confocal 
images) (see claim 73 and 32:66-33:32, Podoleanu). 

Thus, it would have been obvious to one of ordinary skill in the art to have included an 
optical mapping procedure as taught by Podoleanu into Gvili, Luo and Clarke's digital 
camera, as the Podoleanu procedure suggested an optical mapping algorithm with adjustable 
depth resolution and a multiple functionality to ensure that the confocal optical receiver 
operates at the maximum sensitivity (3:62-65, Podoleanu). 
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Regarding claims 36 and 38, these claims recite same limitations as claim 33. Thus they 
are analyzed and rejected as previously discussed with respect to claim 33 above. 

Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Kim et al. (US 2004/0105032) provide a method of enhancing digital image quality 
that enables automatic contrast enhancement by being adaptively performed 
according to a document image having various characteristics input to an image 
processing apparatus including an image input device; 

Cavallaro (US 6,252,632) provides a system for enhancing the television presentation 
of an object by highlighting the object without significantly obscuring the image of 
the object; and 

- Dolgoff (US 7,016,116) provides a viewer with an experience of three dimensional 
images by presenting a composite image source wherein a foreground image and a 
background image which are co-aligned, and employ at least one optical element that 
acts as a beamcombiner. 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
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MONTHS of the mailing date of this final action and the advisory action is not mailed until 
after the end of the THREE-MONTH shortened statutory period, then the shortened statutory 
period will expire on the date the advisory action is mailed, and any extension fee pursuant to 
37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sinh Tran can be reached on 571-272-7564. The fax phone number for the 
organization where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://portal.uspto.gov/external/portal/pair. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free)? If you would like assistance from a USPTO Customer 
Service Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 

/TuanVHo/ 

Primary Examiner, Art Unit 2622 
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